Large-scale city modeling has many important applications, where semi-automatic and automatic image-based modeling methods have been developed. Image-based approaches have the inherent advantage of producing photorealistic textures. However, all these methods suffer from the same weakness. They can only recover the building parts that are visible to a sensor. Invisible regions for a high-rise building by a limited sensor may be inferred from visible regions using prior knowledge. So far, only a few simple prior assumptions such as planarity, smoothness, and rectilinearity have been explored. Resulting models often lack in high level descriptions, which causes the re-usage of these models in content creation applications to be more difficult. Procedural city modeling uses shape grammar rules to generate large scale city models. Applying this to city modeling is interesting, but difficult due to the difficulties of deriving rules for real buildings. In this regard, conventional systems do not allow for the extraction of grammars automatically for buildings or the development of a library of rules, which could be reused in other modeling applications.
It is often not necessary to model a given building “from scratch” as there are existing models available online, for example. The three-dimensional (“3D”) platforms of conventional online platforms for visualizing the Earth, for example, have served large amounts of 3D textured models of real buildings for many cities of the world. These models are reconstructed from aerial images at large-scale to provide a suitable landscape of building tops, however the facade geometry and texture of buildings are in poor quality. Efforts have been made to capture the images at ground level to provide a panoramic view at ground, but there is no 3D geometry reconstructed from these images.
Current reconstruction methods include procedural and image-based modeling methods. Procedural modeling roots in the production system, e.g., Chomsky grammars and shape grammars. Geometry interpretation and control of the derivation are specialized to fulfill the requirement of graphics modeling. An L-system is introduced for plant modeling. For architecture modeling, a computer generated architecture (CGA) shape is introduced by combining the set grammars with a split rule, yielding a powerful deviation system for detailed building geometries. Although the design of grammar systems has been studied, there is only limited work on how to extract the grammars from existing models. For instance, one grammar extraction method employs a top-down partition scheme to extract the split rules from a rectified facade image. However, extracted grammar rules are limited to grid-like subdivisions.
Image-based modeling utilizes images as input to reconstruct the 3D geometrical representation of objects. Structure from motion (SFM) and multiple-view stereo are usually the starting point of image-based modeling. After 3D point clouds are reconstructed, a model regularization process is utilized to finally generate a clean 3D mesh model. The fundamental task of image-based modeling is to infer a structural 3D mesh from the unstructured 3D point clouds. And usually, only partial point clouds can be reconstructed. In contrast to using general surface smoothness assumptions, recently, more and more domain-specific knowledge are incorporated into the inference process according to the problem domain. One method used a branch library extracted from visible branches to synthesize a complete tree model. Another method used a developable surface to approximate dominant facade surface, and used more restricted rectilinear assumptions.
The above-described deficiencies of today's 3D modeling are merely intended to provide an overview of some of the problems of conventional systems, and are not intended to be exhaustive. Other problems with conventional systems and corresponding benefits of the various non-limiting embodiments described herein may become further apparent upon review of the following description.